Inter-rater reliability testing for utilization management staff.

INTRODUCTION Recent regulatory pressures and certification requirements have heightened the need for payer organizations to abide by specific standards regarding medical management operations. Payer organizations that are making medical-necessity determinations regarding reimbursement for health care services, as well as risk-bearing provider groups to whom some of these functions have been delegated, must insure consistency and appropriateness in their determinations. The most powerful accreditation body for health care payers, the National Committee for Quality Assurance (NCQA), requires that payer organizations carry out periodic inter-rater reliability assessments to insure consistency in medical management decision making. An inter-rater reliability assessment or study is a performance-measurement tool involving a comparison of responses for a control group (i.e., the " raters ") with a standard. Inter-rater reliability (also called inter-observer reliability) traditionally refers to how well two or more raters agree and is derived from the correlation of different raters'